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TECHNICAL MEMORANDUM 


SPACE TRANSPORTATION ARCHITECTURE: RELIABILITY SENSITIVITIES 

I. INTRODUCTION 


"... for the 1990's, Space Station Freedom, our critical next step in all our space 
endeavors..." President George Bush, July 20, 1989. 

The article "Fleeing Freedom" 1 points out that along with living in space or on Earth 
comes the chance of injury, illness, and death. One thing the numbers show is that, after 3 to 7 
years of space operation, a rescue mission or vehicle would be needed just to return a crew 
member deconditioned by zero-gravity. The Space Station Freedom project manager says 
"several times in the 30-year life of the station" injury or illness will force return to Earth. He 
further stated that "looking at the history of manned space flight, we've had two failures in space 
that forced returns — Gemini 8's stuck-on attitude thrusters and the Apollo 13 explosion. 

Applying these over the base of U.S. man-hours in space would point to a rescue at the station 
several times a year! Figuring we should be able to do an order of magnitude better comes out to 
once every 2 to 3 years. Over 30 years, this means about 20 rescues." 

A rescue craft, either on-orbit or ground-based, may become a reality even without cost- 
benefit analysis overwhelmingly in its favor. The prospect of all the world seeing the ordeal of a 
stranded crew or a dying crew member nightly on television is chilling. The national nightmare of 
a crew in trouble with no timely way home, no matter what the chances of occurrence, is reason 
enough for many, both within and outside of NASA, to push for a rescue vehicle as a political 
necessity. 

Possibly the most emotional crew risk is that of another catastrophic loss of an orbiter 
and grounding of the shuttle fleet. Apart from the operations trauma, this would deny access to or 
from the space station if another type of vehicle were not available. 

A joint NASA-USAF (United States Air Force) effort proposes several approaches to 
providing a safe and reliable transportation system not only for space station but for other 
concepts of space-based installations. 

The approaches, known as "architectures," combine several types of launch vehicles to 
bring about a transportation system capable of supporting the Space Station Freedom 
construction phase, as well as support of other space-based installations. 

The predominant objective of this report is to investigate one of the proposed architectures. 

A sensitivity analysis model will be developed to reveal the probable occurrences of 
failures, the costs associated with the failures, as well as the life cycle costs related to the 
vehicles that comprise the architecture. In addition, the model should be of value in bringing to 
light a more achievable launch capability of these vehicles. 


II. LITERATURE SURVEY 


NASA’s position on vehicle reliability fluctuates from one engineer to another. Some 
engineers have a very optimistic view of vehicle reliabilities, and others have a very pessimistic 
view. The purpose of this report was not to prove or disprove either view, merely to establish the 
outcome and sensitivity of both views, and how these views affect variables such as launch rate, 
life cycle cost, and cost of failure. A sensitivity analysis is necessary to substantiate these 
results. 

The general purpose of a sensitivity analysis 2 is: 

To identify sensitive parameters, to try to estimate these parameters 
closely, and then to select a solution that remains a good one over the 
range of likely values of the sensitive parameters. 

Sensitivity analysis involves changing one parameter at a time in the original model to check its 
effect on the solution. The changed parameter in this model will be the reliability value associated 
with the Space Transportation System (STS) comprised of four orbiters. 

A simulation model was built to accomplish the sensitivity analysis. A model may be 
used as a representation of a system to be brought into being, or to analyze a system already in 
being. 3 Simulation is defined as an imitation of the operation of a real-world process or system 
over time. 4 The behavior of a system over some specified time can be studied by developing a 
simulation model. The model simply takes on the characteristics in terms of mathematical and 
logical relationships embedded in the system. Once the simulation model has been augmented, 
verified, and validated, a variety of questions can be answered about the real-world system. 
Simulations permit inferences to be drawn about systems by eliminating the need to do the 
following: 5 

1. Build the system. Building the system would be unnecessary especially if it is a 
proposed system. 

2. Disturb the system. Disturbing the system may be unfavorable if the system is 
operational and is costly to experiment with. 

3. Destroy the system. It would be useless to perform a limit of stress test on a system 
that could be destroyed in the process. 

Four of the vehicles analyzed in this report are proposed vehicles, and, although it would 
be helpful to have a prototype of the vehicles to reveal vehicle performance, it would also be 
unprofitable to spend billions of dollars on a vehicle that may not be approved. Therefore, a 
simulation would be the most rational approach. 

The simulation model and sensitivity analysis combination will hereafter be referred to as 
a "sensitivity analysis model." 

The sensitivity analysis model for this report is primarily concerned with five factors. 
These factors are: 
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1. Reliability 

2. Downtime 

3. Effective launch rate 

4. Life cycle cost 

5. Cost per vehicle failure. 


A. Reliability 

The Office of Technical Assessment of the United States Congress speaks of reliability as 
follows: 6 

Reliability is the probability with which a system will perform an intended function. 

A system designed to perform several distinct functions will give a reliability cor- 
responding to each function. For example, a fully reusable vehicle would be designed 
to transport payloads to orbit safely and return safely. The probability that it will 
reach orbit and safely deploy a payload (its ascent reliability) is greater than its 
mission success reliability — the probability that it will reach orbit, safely deploy a 
payload, and return. Mission success reliability is a commonly used criterion, but 
reliabilities of noncritical subsystems are also of interest because they affect main- 
tenance costs .... One of the difficulties in using reliability as a criterion is the uncer- 
tainty in estimates of the reliabilities of operational vehicles and, especially, proposed 
vehicles. 

The many definitions of reliability that exist depend upon the viewpoint of the user. However, 
they all have a common core that contains the statement that reliability, R(t), is the probability 
that a device performs adequately over the time interval [0,r]. The device under consideration 
may be an entire system, a subsystem, or a component. 2 


B. Downtime 

Downtime is the time the system is nonoperational following a system failure. Downtime 
can also be described as the time required to repair the system after a failure has occurred. 


C. Effective Launch Rate 

The effective launch rate (ELR) is the actual launch rate. All vehicles will have a 
prescribed or nominal launch rate. This prescribed launch rate is mandated by the flight manifest 
(table 1). However, by subjecting these vehicles to a probabilistic environment, the prescribed 
launch rate will not be achieved. Therefore, the flights actually launched become the ELR. 


3 
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D. Life Cycle Cost 


Life cycle cost refers to all costs associated with the system or product as applied to the 
defined life cycle. Life cycle cost is determined by identifying the applicable functions in each 
phase of the life cycle, costing these functions, applying the appropriate costs by function on a 
year-to-year schedule, and ultimately accumulating the costs for the entire span of the life cycle. 3 
It should be noted that all life cycle costs may be difficult (if not impossible) to predict and meas- 
ure. For instance, some indirect costs caused by the interaction effects of one system on another, 
social costs, and so on, may be impossible to quantify. Thus, the emphasis should relate pri- 
marily to those costs that can be directly attributed to a given system or product. 3 A life cycle 
cost breakdown is presented in table 2. 

Table 2. Cost categories. 3 

1. Research and development 

(a) Program development 

(b) Advanced research and development 

(c) Engineering design 

(d) Equipment development and test 

(e) Engineering data 

2. Investment 

(a) Manufacturing 

(b) Construction 

(c) Initial logistic support 

3. Operations and maintenance 

(a) Operations 

(b) Maintenance 

-Maintenance personnel and support 
-Spare/repair parts 

-Test and support equipment maintenance 
-Transportation and handling 
-Maintenance training 
-Maintenance facilities 
-Technical data 

(c) System/equipment modifications 

(d) System phase-out and disposal 


E. Cost Per Vehicle Failure 

The cost per vehicle failure is simply the cost associated with each vehicle failure. This 
cost includes the cost of losing flight hardware and payload. 
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III. PROBLEM DESCRIPTION 


For permanent human presence in space, NASA has taken a close look at the ramifica- 
tions associated with humans living and working in space. Of course, one demand of humans 
residing in space is food and water. Items for personal hygiene and health maintenance are also 
necessary. 

Currently, the only space transportation system available for any type of emergency res- 
cue or logistics delivery is NASA's STS. However, studies have been accomplished to examine 
the potential of STS performing these functions. One study 7 determined that the reliability of the 
shuttle system for support of the space station may not adequately support the deployment 
phases and possible day-to-day support required by a permanently manned space station. Also, 
having a manned vehicle to carry out all the flight requests of space station greatly increases the 
risks associated with manned space programs. Still other possibilities exist when there may be 
periods in the shuttle program when shuttle rescue missions are not possible. For example: 

- There may not be sufficient orbiters or boosters in inventory to have the shuttle cycling 
fast enough to be ready for a rescue mission at a short notice. 

- The shuttle may be grounded for a time, in the same way that aircraft are grounded from 
time to time. 

In November 1989, President Bush reaffirmed a National Space Policy that established 
the following goals: 8 

1. Strengthen the security of the United States. 

2. Obtain scientific, technological, and economic benefits. 

3. Encourage continuing U.S. private sector investment. 

4. Promote international cooperative efforts. 

5. Expand human presence and activity beyond Earth orbit into the solar system. 

6. Assure access to space, sufficient to achieve all U.S. space policy goals. This is the 
key element of the National Space Policy. 

7. U.S. space transportation systems must provide a balanced, robust, and flexible 
capability with sufficient resiliency to allow continued operations despite failures in a 
single system. 

One of the National Space Policy guidelines states that the National Space Transportation 
capability will be based on a mix of vehicles consisting of the current STS, unmanned launch 
vehicles (ULV's), and in-space transportation systems. 

As a result of this National Space Policy, NASA and the USAF launched a project to 
review the Nation’s space transportation capability. The review was conducted in three phases: 
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Phase I - Define current launch situation. 


Phase II - Identify alternatives to resolve capabilities/needs mismatches. 

Phase III - Assess policy, procedure, and investment actions to meet U.S. space launch 
objectives. 

Several architectures were developed as possible candidates for meeting the aforemen- 
tioned goals. The architecture analyzed in this report (fig. 1) is one of the several NASA chose to 
investigate. Architecture is defined as any design or orderly arrangement perceived by man. 9 

Since the life expectancy of a launch vehicle is about 30 years, each architecture has a 
flight manifest (table 1) of 30 years. A vehicle data sheet for each vehicle is provided in appendix 
A. 


The architecture is divided into four categories: cargo vehicles, new elements, manned 
vehicles, and facilities. It is also divided into six 5-year time frames beginning in 1990 and ending 
in the year 2020. 


A. Cargo Category 

The Titan IV vehicle would fly a prescribed number of flights each year from 1990 through 
2020. The ET/Core 1.5 (External Tank Derived Vehicle/ 1.5 Stage) would come on line in the year 
2000 and fly its prescribed flights. The other vehicles would come on line as indicated and do 
likewise. The ET/Core vehicles will hereafter be referred to as ETD1 (ET/Core 1.5 stage), ETD2 
(ET/Core with 2 core-derived booster), and ETD3 (ET/Core with 3 core-derived booster). 


B. New Elements Category 

An advanced solid rocket motor (ASRM) would be available for integration into the 
transportation system by the year 1995. At that time, the STS would use both the new element 
(ASRM) and the existing element redesigned solid rocket motor (RSRM) for the duration of the 
architecture. The space transportation main engine (STME) would come on line at the desig- 
nated time for use with the ET-derived vehicles. The use of dual elements is an attempt to 
provide resiliency to the architecture. Resiliency is defined as the ability of an STS to adhere to 
launch schedules despite failures — to "spring back" after failure. 6 


C. Manned Category 

The STS, as referenced in the previous category, would fly with RSRM's until the ASRM's 
came on line, at which time the SRM's would be interchangeable. The STS is to be used for both 
personnel and cargo delivery to the space station. The personnel launch system (PLS) would 
come on line in the year 2000 and utilize the ETD1 as its launch vehicle. The primary purpose of 
the PLS is to transfer personnel to and from Space Station Freedom and to serve as an emer- 
gency rescue vehicle if necessary. However, when the shuttle system (STS) experiences a 
failure or is in the nonoperational state, both PLS and ETD1 flight rates would be increased to 
accommodate the number of proposed shuttle personnel and cargo launches. However, STS 
failures that occur prior to these vehicles coming on line will not have replacement flights. 
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D. Facilities Category 


This category simply identifies the new facilities required to support the proposed archi- 
tecture and indicates when these facilities would be required. 

Although the architecture consists of six vehicles, only five would be used for space 
station support. These are: STS, PLS, ETD1, ETD2, and Titan IV. ETD3 is designated as a 
lunar/Mars support vehicle. The lunar/Mars project will not be addressed in this report. 

The questions to be answered are: 

1. How resilient is this system? 

2. What costs are associated with the life of the system as well as with vehicle failures? 


IV. DATA ACQUISITION AND GENERATION 


This section discusses the acquisition and generation of the data necessary for the 
sensitivity analysis model. The variables of interest are as follows: 

1. Vehicle Flight Rates 

2. Vehicle/Subsystem Reliabilities and Downtimes 

3. Vehicle Operation Time 

4. Costs 

(a) Design, Development, Test, and Evaluation (DDT&E) 

(b) Ownership 

(c) Lost Payload 

(d) Lost Hardware (vehicles) 

(e) Flight. 


A. Vehicle Flight Rates 

The vehicle flight rates were obtained from NASA during the initial definition of the archi- 
tectures. 
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B. Vehicle/Subsystem Reliabiity Estimation and Downtimes 

One of the purposes of this report is to determine the sensitivity of the architecture to 
vehicle reliabilties. As stated in section III, one of the difficulties in using reliability as a criterion 
is the uncertainty in estimates of the reliabilities of operational vehicles and, especially, proposed 
vehicles. 

An OTA publication 10 had this to say about "reliability estimation"... 

The most difficult and least credible part of this procedure is estimating 
the probability of failure for each vehicle. This is particularly true for pro- 
posed vehicles that have not been fully designed, much less built, tested, 
and flown. The only completely objective method of estimating a vehicle's 
probability of failure is by statistical analysis of the number of failures 
observed in actual launches of identical vehicles under conditions repre- 
sentative of those under which future launches will be attempted. 

The design reliability of proposed vehicles is generally estimated using: 10 

• Data from laboratory tests of vehicle systems (e.g., engines and avionics) and com- 
ponents that have already been built 

• Engineers' judgments about the reliability achievable in systems and components that 
have not been built 

• Analyses of whether a failure in one system or component would cause other systems 
and components, or the vehicle, to fail 

• Assumptions (often tacit) that: 

- The laboratory conditions under which systems were tested precisely duplicate con- 
ditions under which the system will operate 

- The conditions under which the systems will operate are those under which they 

were designed to operate 

- The engineers' judgments about reliability are correct 

- The failure analyses considered all circumstances and details that influence reliability. 

Such "engineering estimates" of design reliability are incomplete and subjective. However, the 
subjectivity and uncertainty often are not exhibited. There are methods for assessing and exhib- 
iting the uncertainties of experts called upon to estimate reliabilities of components, and prob- 
abilistic risk assessment (PRA) methods for estimating risks posed by unreliability, considering 
the uncertainties in the estimates of components’ reliabilities. However, it is more difficult and 
time-consuming to use them than to provide a single "best estimate" of reliability showing no 
uncertainty. The latter has been standard engineering practice except for tasks — such as safety 
analyses of nuclear reactors— for which the increased rigor has been deemed worth the effort. 10 
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The shuttle occupies a commanding position over other existing vehicles and, therefore, 
was selected as the vehicle on which the proposed vehicle reliabilities will be contingent. In other 
words, the space shuttle main engine (SSME) reliability will be the assumed STME reliability, 
and the shuttle avionics & other reliability will be the assumed proposed vehicle avionics & 
other reliability. 

The actual reliability of the shuttle system is unknown, but may lie between 97 and 99 
percent. 11 For the purpose of the sensitivity analysis, the values of 0.97, 0.98, and 0.99 were 
selected for the reliability variable. These reliabilities were then allocated to the various vehicle 
subsystems in two categories: common failures and unique failures (table 3). This allocation 
procedure was developed by Dr. James W. Steincamp, NASA-MSFC, Chief, Operations 
Analysis Branch/Preliminary Design Office. 12 Appendix B contains a detailed description of this 
allocation procedure. A common subsystem failure affects all vehicles that share the failed sub- 
system. This means that any vehicle requiring the failed subsystem is blocked until the failure 
has been repaired or until the failed subsystem is flight ready. A unique subsystem failure affects 
only the vehicle on which the failure occurred. Any other vehicle requiring that subsystem does 
not get blocked and is allowed to launch. Section V will discuss in detail the activity of common 
and unique subsystem failures. 

Titan IV failure probability was obtained from an OTA special report. 10 The OTA relia- 
bility estimate for Titan IV is 96.2 percent. The OTA reliability estimate for STS is 96.6 percent. 
With that information, a relationship must be developed between STS and Titan IV so that as 
STS reliability varies, Titan IV reliability can vary conformably. This can best be demonstrated as 
follows: 


0.962 _ * (Titan IV) 

0.966 0.970 (STS) 

x = 0.966 . 

Therefore, the relationship between STS and Titan IV can be presented as in table 4. 

Since the Titan IV vehicle is independent of the other vehicles in the architecture (its 
failure or success has no effect on any other vehicle), its failures were not categorized. Once a 
failure occurred, the vehicle is nonoperational for some specified time. The downtimes and 
probability of occurrence for the Titan IV vehicle were obtained from L Systems, Inc., El Segundo, 
California. 6 This data is as shown in table 5. 

Downtimes (with the exception of Titan IV) were also categorized into common and 
unique. The triangular distribution generated the downtimes in the sensitivity analysis model. 
The triangular distribution is utilized when a most likely value can be ascertained along with 
minimum and maximum values, and a piecewise linear density function seems appropriate. 5 
Figure 2 gives the density function for the triangular distribution and its graph. 

For common failure downtime, the values of a (minimum), m (mode), and b (maximum), 
are equal to 273.75 (0.75*365), 365.0, and 638.75 (1.75*365) (days), respectively. Since the 
common failures correspond to subsystems shared by several vehicles (e.g., SSME, avionics & 
other), a minimum of 9 months and a modest chance of exceeding 18 months seems appropriate, 
with a slight bias toward the shorter downtimes (fig. 3). 12 
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Table 3. Reliability allocations 


STS 

Reliability 

Odds Mai 

COMMON ELEMENTS 
mp8 (3ssme) 

0.9904 

104 

2 x srb 

0.9900 

100 

avion ic* & other 

0.9962 

266 

TOTAL 

0.9767 

43 

UNIQUE ELEMENTS 
avionics 4 other 

0.9970 

332 

operations 

TOTAL 

0.9962 

0.9932 

266 

148 

STS SYSTEM 

0.9701 

33 

P(COMMON) 

PjUNIQUE) 

0.7748 

0.2252 

AM 

129 

P(ssme) 

0.41 

1.71 

as 

0.43 

0.16 

1.76 

1.19 

PLS/ETD1 



COMMON ELEMENTS 
mps (5stme w 2sus) 

0.9935 

154 

av onica & other 

0.9962 

266 

TOTAL 

0.9898 

98 

UNIQUE ELEMENTS 
avionics & other 

0.9970 

332 

operations 

0.9962 

266 

TOTAL 

0.9932 

148 

PLS/ETD1 SYSTEM 

0.9831 

59 

P(COMMON) 

P(UNIQUE) 

0.6016 

0.3984 

2.51 

1.66 

P(stme) 

0.63 

2.73 

Pjaio) 

0.37 

1.58 

ETD2 



COMMON ELEMENTS 


13 

mps (17stme) 
avionics & other 

0.9235 

0.9962 

266 

TOTAL 

0.9200 

13 

UNIQUE ELEMENTS 
avionics & other 

0.9970 

332 

opergions 

TOTAL 

0.9962 

0.9932 

266 

148 

ETD2 SYSTEM 

0.9138 

12 

P(COMMON) 

0.9220 

12.82 

P(UNKXJE) 

0.0780 

1.08 

Prttme) 

0.96* 

23.00 

P(aAo) 

0.05 

1.05 

ETD3 



COMMON ELEMENTS 



mps (24 stme) 
avionics & other 

0.9093 

0.9962 

11 

266 

TOTAL 

0.9059 

11 

UNIQUE ELEMENTS 
avionics & other 

0.9970 

332 

operations 

TOTAL 

0.9962 

0.9932 

266 

148 

ETD3 SYSTEM 

0.8998 

10 

P(COMMON) 

P(UNIQUE) 

0.9329 

0.0671 

14.91 

1.07 

P(stme) 

0.96 

27.48 

P{aAo) 

0.04 

1.04 


• >1 (Round-oft Error) 


0.08 0.09 


Rallabilfcv 

Odds Mart 

ReliatolBtY 


0.9936 

156 

0.9968 

313 

0.9933 

150 

0.9967 

300 

0.9975 

400 

0.9988 

800 

0.9645 

65 

0.9922 

129 

0.9980 

500 

0.9990 

1000 

0.9975 

400 

0.9988 

800 

0.9955 

222 

0.9978 

445 

0.9601 

50 

0.9900 

100 

0.7751 

4.45 
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Figure 4. Unique failure downtime assumption. 

For unique failure downtimes, the values of a, m, and b are equal to 91.25 (0.25*365), 
182.5 (0.5*365), and 547.5 (1.5*365) (days), respectively. Since the unique failures correspond 
to subsystems not shared by other vehicles (e.g., landing gear, aerosurfaces, and actuators), the 
choice of 6 months as a mode reflects a belief that most (80 percent) of the downtimes will last 
less than a year, and only a few (5 percent) will last longer than 15 months (fig. 4). 12 


C. Vehicle Operation Time 

The time of vehicle operation (years) was determined by the vehicle manifest (table 1). It 
is viewed as the year of the first launch to the year of the last launch (even if there are zero 
flights in a year somewhere in between). 


D. Costs 

All costs entered into the model were provided by the Engineering Cost Group/Prelimin- 
ary Design Office, NASA-MSFC.^ However, government-developed costs for proposed 
vehicles are "sensitive." Consequently, the empirical data was coded to preserve the sensitive 
nature of the data. 
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The definition of these costs are as follows: 


DDT&E — All costs associated with the actual design, development, testing, and evalua- 
tion of the vehicle. 

Ownership — The base cost of owning the vehicle. 

Lost Payload — The payload cost incurred when a failure occurs. 

Lost Hardware — The cost of the lost vehicle due to failure. 

Flight — All costs associated with launching the vehicle. 

V. MODEL DEVELOPMENT 


After subjecting a nominal launch schedule to a probabilistic environment, what would be 
the achieved launch rate? How would the failures and downtimes affect costs? How resilient 
would the architecture be? Questions like these can be answered after simulation runs of the 
sensitivity model have been accomplished. 

The components of the model are as follows: 

1. Representation of the architecture 

2. Variables used in the execution of the model 

3. Data input to the model 

4. Actual runs of the model 

5. Tabulation of the model results. 


A. Simulation Language 

The simulation language SLAM (Simulation Language for Alternative Modeling) devel- 
oped by A. Alan B. Pritsker and C. Dennis Pegden was utilized for the purpose of building this 
model. SLAM is a widely used simulation language capable of modeling a variety of systems. 
With SLAM, systems can be depicted using a combination of network symbols, discrete events, 
or continuous representations. This flexibility permits one to accurately model virtually any 
process, including manufacturing operations, transportation systems, communication networks, 
computer systems, military operations, and material handling systems. Symbols, rather than 
complicated commands, provide the framework for all model building. Most important perhaps is 
the ability to build, simulate, analyze, compare, and present models of a wide variety without 
ever leaving SLAM (SLAMSYSTEM). 
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B. Basic Assumptions 


Simulating a system such as this is a complicated procedure, especially when several of 
the vehicles are only proposed. It is unlikely that every possibility and condition can be taken into 
account. 

A number of assumptions have been made in the development of this model. Assumptions 
were made to avoid an abundance of detail that would obstruct the true goals of this model. The 
following assumptions have been made concerning the vehicles that will be studied in this model: 

1. Only subsystem downtimes can affect vehicle availability. 

2. There will always be enough subsystems (boosters, cores, engines, etc.) available 
for integration. 

3. Once all subsystems required by a particular vehicle are available, the vehicle will 
attempt launch. 

4. Ground operation tasks of preparing the vehicles are assumed effective and are not 
modeled. 

5. Any failure of major subsystems at launch is catastrophic. 

6. All vehicles except STS are produced at a rate that would not affect the flight rate 
even if a failure occurs. 

7. Downtimes are assumed triangularly distributed. 

8. STME reliability is assumed to be the same as SSME reliability. 

9. The STS is comprised of four orbiters (initially). 

10. No time value of money considerations (cost in 1990 constant dollars). 


C. Data Input and Initialization 

Before executing the model, the user must input various information such as the vehicle 
reliability, probability of common and unique failures, reliability of subsystems, and the prescribed 
flight rates. 

The reliability of the PLS and three ETD vehicles are derived from the reliability of STS, 
while Titan IV reliability was obtained from an OTA publication (see section IV). Depending 
upon the assumed reliability of STS (0.97, 0.98, and 0.99), the other vehicles will take on 
reliability values comparable to STS using the same subsystem reliabilities assigned to STS 
subsystems. 
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D. Model Characteristics 


Basic to the description of model characteristics are the following variables: 

1. Flight Requests — Flight requests are generated based on the flight manifest (table 1). 
A flight request does not guarantee a vehicle launch. 

2. Launch Authorization — All vehicles have one launch authorization. Once the launch 
authorization has been issued, and all subsystems are operational, the vehicle can attempt 
launch. The launch authorization for any vehicle is not issued again until the current vehicle (in 
possession of the launch authorization) either launches successfully, in which case the launch 
authorization is returned to the launch authorization center, or until the vehicle/subsystem 
experiences downtime (in the case of a failure) and is again flight ready. 

3. Launch Authorization Center — The launch authorization center is where the flight 
requests must wait for launch authorization. All vehicles have one launch authorization center. 

4. Vehicle — A vehicle is the entity that makes the launch attempt. It is the union of a 
flight request and a launch authorization. 

Certain criteria must be met before a vehicle attempts a launch: 

1. All subsystems must be operational 

2. The launch authorization must be issued. 

In addition to the above criteria, STS must also have an available orbiter. 

A flight request is made to wait in the launch authorization center when: 

1. The launch authorization is being used by another vehicle at the time of the flight 
request arrival. 

2. A common subsystem has failed and has thereby halted flights of any vehicle requiring 
the subsystem (the launch authorization has not been released). 

STS flight requests do not wait but are immediately routed to the PLS and ETD 1 launch authori- 
zation centers where they are given priority over any waiting PLS and ETD1 flight requests. This 
is done to ensure that all STS personnel and cargo flights are launched. STS is the primary space 
station support system. The other vehicles alleviate the flight demands placed on STS. 

The general flow of the sensitivity analysis model is as follows: 

1 . The flight request arrives at the launch authorization center. 

2. The flight request resides at the launch authorization center until a launch 
authorization is available. If a launch authorization is available upon the arrival of the flight 
request, it seizes the launch authorization. 

3. The vehicle (flight request plus launch authorization) now attempts launch. 
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(a) Following a successful launch, the launch authorization is released and returned 
to the launch authorization center. For STS, an orbiter is released as well. 

(b) Following an unsuccessful launch, the launch authorization is not released until 
the failure cause has been determined and repaired (downtime). For STS, the 
current orbiter is destroyed and a new orbiter must take its place. The time 
required to replace the orbiter is approximately 5 years. 11 

4. The launch authorization having been returned to the launch authorization center, the 
flight request exits the system. 

A flowchart of the sensitivity model is presented in figures 5, 6, 7, and 8. 



Figure 5. Sensitivity analysis model (Titan IV flow). 
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Figure 7. Sensitivity analysis model (ETD vehicle flow). 
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Figure 7. Sensitivity analysis model (ETD vehicle flow) (continued) 
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Figure 8. Sensitivity analysis model (common to all vehicle flows). 


E. Model Output 

The sensitivity analysis model is evaluated by examining the output of the model. The 
output aids the user in determining the accuracy of the model's performance; and in this analysis, 
it allows the user to recognize the sensitivity of the model to various changes in the data input. 

The SLAM II output summary report includes the following: 

1. General information such as project name, modeler name, date, run number, and current 
time at end of run. 

2. Statistics for selected collection points including mean, standard deviation, coefficient 
of variation, minimum values, maximum values, and number of observations. 


I 
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3. File statistics for queues/ AW AIT nodes such as file number, file label, file type, 
average length, standard deviation, maximum length, current length, and average wait time. 

4. Regular activity statistics such as index/label, average utilization, standard deviation, 
maximum utilization, current utilization, and entity count. 

5. Service activity statistics such as activity number, activity label or start node, server 
capacity, average utilization, standard deviation, current utilization, average block, maximum idle 
time per server, maximum busy time per server, and entity count. 

F. Model Versatility 

The model is structured such that the user may input any flight manifest for the vehicles. 
This feature is useful in determining the sensitivity of the model to increased or decreased flights 
per vehicle. The user may also decrease the number of vehicles in the model. The user can 
change the number of SRM's from two (dual subsystem) to one. This is useful to study the 
effects of having dual versus single boosters. The user can change the downtime length to 
discover the implication of shorter versus longer downtimes. 

G. Model Limitations 

Due to the structure of the NLS architecture, the maximum vehicle types allowed in the 
model is six. 

Uncertainty in connection with the actual reliability of existing vehicles introduces addi- 
tional limitations to the model. Allowing the reliability to be determined by a random process, 
rather than using assigned reliability values (0.97, 0.98, 0.99), would enhance the real world 
portrayal of the model. 


VI. MODEL VALIDATION 


Before the end results of a developed simulation model are analyzed, the simulation 
program needs to be validated. It is necessary to determine if the program sufficiently simulates 
real world occurrences. This entails making a comparison between the results of the program and 
observed data in the real world. 4 

Validation is the determination that a model is an accurate representation of 
the real system. Validation is usually achieved through the calibration of the 
model, an iterative process of comparing the model to actual system behavior 
and using the discrepancies between the two, and the insights gained, to 
improve the model. 4 

The basis of this report is a sensitivity analysis model. There is no real-world data with 
which to compare program results. Without any real data as a standard of comparison, the only 
way to validate the overall model is to have knowledgeable people carefully check the credibility 
of output data for a variety of situations.^ It is virtually impossible to prove that any model is 
100-percent valid; however, there are ways to demonstrate a high level of validity. 
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Two sets of statistics that can give a quick indication of model reasonableness are 
"current contents" and "total count." These statistics apply to any system having items of some 
kind flowing through it, whether these items are called customers, transactions, inventory, or 
vehicles. Current contents refers to the number of items in each component of the system at a 
given time. Total count refers to the total number of items that have entered each component of 
the system by a given time. 4 The SLAM summary report produces both statistics. For purposes 
of validating the sensitivity analysis model, the statistic of interest will be total count. 

The method chosen to evaluate the model will be to run the model for extreme conditions. 
One such extreme condition is vehicle reliabilities of 100 percent. At 100 percent reliability, one 
would expect no failures to occur and the effective launch rate to equal the nominal launch rate. 
Another extreme condition worth consideration is to eliminate downtimes, or to introduce a 
downtime so small that it would have little or no effect on the model. For example, if the 
downtimes are assigned as zero you would expect all flight requests to be launched since the 
launch authorization cannot be delayed by downtimes and therefore would always be available. 
However, this does not mean that all launches will be successful. 


A. 100-Percent Vehicle Reliability 

Table 6 shows the results of 100-percent vehicle reliability. Results readily identified are 
the failure values recorded. All vehicle failure observations are listed as "No Values Recorded." 
This means that there were no failures. The effective ELR observation equals the nominal launch 
rate. To obtain the SLAM II ELR values, the total number of flights from the manifest must be 
divided by the number of operational years for each vehicle. For example, Titan IV has a nominal 
launch rate of 5.58 flights per year (173 flights in 31 years). From column 7 of table 6, Titan IV 
shows 5,190 flights; 5,190 flights in 30 years equals 173 flights per year and 173 flights in 31 
years equals 5.58 flights per year. Therefore, because the launch rate at 100-percent vehicle 
reliability does generate the anticipated "perfect" run results, the 100-percent vehicle reliability 
condition adds validity to the model. 

Table 6. 100-percent vehicle reliability case. 

SLAM II SUMHARY REPORT 


SIMULATION PROJECT THESIS BY WILLIAMS 

DATE 2/11/1991 RUN NUMBER 30 OF 30 


CURRENT TIME .1 131E+05 

STATISTICAL ARRAYS CLEARED AT TIME .OOOOE+OO 


* ‘STATISTICS FOR VARIABLES BASED ON OBSERVATION* * 


( 1 ) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 


MEAN 

STANDARD 

COEFF. OF 

MINIMUM 

MAXIMUM 

NO. OF 


VALUE 

DEVIATION 

VARIATION 

VALUE 

VALUE 

OBS 

TIT SUCC 
TIT FAIL 

. 87 0E>02 

• 499E+ 02 

. 574E+00 
NO VALUES 

. 1 OOE+O 1 
RECORDED 

. 173E+03 

5190 

STS SUCC 
STS FAIL 

. I51E+03 

. 8 69E+02 

. 575E+00 
NO VALUES 

.100E+0I 

RECORDED 

„ 30 1 E 1-0 3 

9030 

PLS SUCC 
PLS FAIL 

.05OE+OI 

. 461E+01 

. 543E+00 
NO VALUES 

. 1 00E+0 1 
RECORDED 

. 160E+02 

480 

ETDI SUCC 
ETDI FAIL 

. 69 5Et 02 

- 398E+02 

- 573E+00 
NO VALUES 

• 1 OOE+O 1 
RECORDED 

. 1 3BE+03 

4 1 40 

ETD2 SUCC 
ETD2 FAIL 

. 1 B 5 E I 02 

. 1 04 E l 02 

.562E+00 
NO VALUES 

• 1 OOE+O 1 
RECORDED 

. 36OE+02 

1080 

ETD3 SUCC 
ETD3 FAIL 

. 120E+02 

. 664E+G 1 

•553E+0Q 
NO VALUE8 

• 1 00E + 0 1 
RECORDED 

. 230E+02 

690 
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B. Zero Downtime 


A smaller number of runs (10) was necessary for this case in order to include sections of 
the summary report. 


If the model is performing as expected, 10 runs of the model with zero downtimes should 
generate vehicle launch attempts as follows: 1,730 for Titan IV, 3,100 for STS, 160 plus any STS 
failure offloads for PLS, 1,380 plus any STS failure offloads for ETD1, 360 for ETD2, and 230 for 
ETD3. The success or failure values recorded in table 7 (column 7) plainly show that all flight 
requests were honored for Titan IV, STS, ETD2, and ETD3; however, this is not readily identi- 
fied with the PLS and ETD1 recorded values. The summary report provides data that will sub- 
stantiate the postulation that all PLS and ETD1 flight requests were honored. 

Table 7. Zero downtime case. 

SLAM II SUM MA R T REPORT 


SIMULATION PROJECT THESIS 

BY WILLIAMS 



DATE 2/11/1991 

RUN NUMBER 

10 OF 

10 

CURRENT TIME . 1131E+05 

STATISTICAL ARRAIS CLEARED AT TIME 

. OOOOE+OO 




* ‘STATISTICS FOR VARIABLES BASED OH OBSERVATION** 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 


MEAN 

STANDARD 

COEFF. OF 

MINIMUM 

MAXIMUM 

NO. OF 


VALUE 

DEVIATION 

VARIATION 

VALUE 

VALUE 

OBS 

TIT SUCC 

. 85 3E+02 

. 490E+02 

• 5 74 E+00 

. 100E+01 

. 172E+03 

1696 

TIT FAIL 

. 259E+0 1 

• 1 56E+0 1 

• 602E+00 

• 1 OOEi0 1 

• 700E+0 1 

34 

STS SUCC 

. 1 466+03 

. 853E+02 

. 576E+00 

* 100E+01 

. 298E+03 

2955 

STS FAIL 

. 365E+0 1 

. 234E+0 1 

•64 1 E+00 

. 100E+01 

• 1 1 0E+02 

55 

PLS SUCC 

. 100E+02 

. 556E+0 1 

• 556E+00 

* 100E+01 

• 2 1 0E+02 

189 

PLS FAIL 

. 1 O0E+0 1 

. OOOE+OO 

• 000E+00 

. 100E+01 

. 100E+01 

1 

ETD 1 SUCC 

. 708E+02 

• 406E+02 

. 574E+00 

. 100E+01 

. 143E+03 

1405 

ETD 1 FAIL 

* 171E+01 

. B25E+00 

. 481E+00 

. 100E+01 

.300E+Q1 

14 

ETD2 SUCC 

. 174E402 

. 98 1 Ef 0 1 

. 563 E+00 

. t00E+01 

. 360E+02 

338 

ETD2 FAIL 

. 195E401 

. 999E+00 

• 5 1 1 Ef 00 

. 1 0OE+ 0 1 

• 4 OOEfO 1 

22 

ETD3 SUCC 

. 1 12E402 

. 620E+0 1 

. 555E+00 

. 1 OOEfO 1 

. 230E+02 

213 

ETD 3 FAIL 

. 165E+01 

. 786E+00 

. 477E+00 

. 100E+01 

. 300E+0 1 
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Section III explained that for every STS flight request when STS is nonoperational or 
when STS has experienced a failure, two flights are offloaded; one to PLS and one to ETD1. 
Section III further explained that the ETD1 and PLS do not become available until 2000 and 2003, 
respectively. For that reason, the absence of flights in the total count for these vehicles can be 
attributed to failures that occurred prior to the ETD1 and PLS coming on line. Activities of 
interest (see "Activity Index/Label" column) are numbers 49, 50, and 91. Activity 49 represents 
the number of flights per run offloaded to PLS, activity 50 represents the number of flights per run 
offloaded to ETD1, and activity 91 represents the total number of attempted offloads. These 
values are tabulated in table 8. 


Ignoring the initial operational dates for PLS and ETD1, the 55 STS failures, as identified 
in column 7 of table 7, would increase the number of flights of the PLS and ETD1 vehicles from 
160 to 215 for PLS and from 1,380 to 1,435 for ETD1. The summary report of table 7 shows 190 
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Table 8. Tabulation of STS offloads. 


Offloads Offloads Total 

ToPLS ToETDl Offloads 

RunJ (Act #491 (A ct #5 0) (A ct # 911 


1 


5 


5 


5 


2 


2 


3 


6 


3 5 


6 


7 


4 2 


2 


5 


5 4 


4 


5 


6 3 


4 


5 


7 5 


8 


11 


8 


2 


3 


4 


9 


3 


10 1 3 4 

Total 30 39 55 


(189 successes, 1 failure) PLS flights and 1,419 (1,405 successes, 14 failures) ETD flights. This 
yields a delta of 25 (215-190) PLS flights and 16 (1,435-1,419) ETD1 flights. However, as table 
8 reveals, of the 55 attempted offloads, only 30 were successfully offloaded to PLS and 39 to 
ETD1, thus accounting for the deltas of 25 and 16 since 30+25=55, and 39+16=55. The 
unsuccessful offloads occurred prior to the PLS and ETD1 operational dates. Therefore, as 
expected, all launch requests were honored. Because the anticipated launch attempts are equal 
to the number of flight requests the zero downtime condition also adds validity to the model. 

VII. RESULTS 


This analysis closely examines the implications of vehicle reliabilities ranging from 0.97 to 
0.99. The analysis involved determining the effective or expected launch rates (ELR) for all 
vehicles in the architecture and the costs connected with the failure (CF) and life (LCC) of those 
vehicles. The tools used were observed vehicle failure probabilities and downtimes, data based 
on engineering judgment, and the development of a sensitivity analysis program. 

The ELR was calculated as follows: 


ELR = number of launch attempts/horizon (flights per year) 

where horizon is defined as the number of operational years of the vehicle. The launch attempts 
are summed by the program. 
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The CF was calculated as follows: 

CF = (COP * APC) * (number of failures) ($B) 

CP 

where CP is the cost of payload due to a failure ($B), COP is the cost of lost payload ($k/lb), and 
APC is the adjusted payload capability (100 klb/flight). This equation is modified by adding a 
vehicle manufacturing cost for STS and PLS to accommodate the loss of a reusable vehicle. The 
other vehicles are considered expendable and do not require any additional cost considerations. 

The LCC was calculated as follows: 

LCC = CPF * (number of attempts) + CBH * horizon + CF + DDTE ($B) 

where CPF is the cost per flight ($B/flight), CBH is the cost of vehicle ownership ($B/year), and 
DDTE is the design, development, testing, and evaluation phase of the vehicle's life ($B). 

The sensitivity analysis program allows the user to gather information about each 
vehicle’s expected launch rate, or the actual number of flights launched. The results show (fig. 9), 
as expected, that the effective launch rate is directly related to vehicle reliabilities. As vehicle 
reliabilities increase, the effective launch rates increase. Note, however, that as the shuttle 
reliability increases, the PLS and ETD1 ELR's decrease. This is due to the elimination of 
offloading to these vehicles caused by shuttle failures. In the section III discussion on manned 
vehicles, it was noted that STS offloads flights when a failure occurs, or if a flight is requested 
and the STS system is not operational. As a result of that, as STS becomes more reliable, the 
lower flight rates will be for the PLS and ETD1 vehicles; and conversely, as STS becomes less 


NOTE: 30 RUNS OF 31 YEARS DURATION 
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VEHICLE 

Figure 9. Effective launch rates. 
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reliable, the higher the flight rates will be for the PLS and ETD1 vehicles. The vehicles that are 
not assigned very demanding flight rates (ETD2 and ETD3) are really not gaining anything by 
having a higher reliability. Both ETD2 and ETD3 are averaging approximately the same ELR for 
all three cases (0.97, 0.98, and 0.99). Also, the Titan IV vehicle, which operates independent of 
other vehicles, does not show improvement or decline with separate reliability assignments. 

The results cited thus far all have to do with the launch rates of the vehicles. However, 
costs are closely connected with launch rates. The cost accrued with vehicle failures, as well as 
the life cycle costs, can be accessed by the user without difficulty. 

As expected, costs are inversely related to vehicle reliabilties. The more reliable the 
vehicle, the less the costs accrued over the life of the vehicle, as well as the costs due to vehicle 
failure. Note that both the cost of failure and life cycle cost values are decreasing as the reliability 
of the vehicles increase. Figure 10 demonstrates the inverse relationship between cost of failure 
and reliability. Figure 1 1 demonstrates the inverse relationship between life cycle cost and 
reliability. A consolidation of figures 9 and 11 generated figures 12 to 17. For each vehicle, these 
figures demonstrate both ELR and LCC sensitivity to the reliability range 0.97 to 0.99. 

Other results obtainable at the end of each run are: 

1. Number of failures of a specific type (SSME, ASRM, avionics, etc.) 

2. Number of flight attempts, successes, and failures. 

3. Number of attempted and actual offloaded flights. 
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Figure 10. Cost of failure. 
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Figure 11. Life cycle cost. 
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Figure 12. TIV ELR and LCC sensitivity to reliability. 
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Figure 13. STS ELR and LCC sensitivity to reliability. 
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Figure 14. PLS ELR and LCC sensitivity to reliability. 
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Figure 15. ETD1 ELR and LCC sensitivity to reliability. 
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Figure 16. ETD2 ELR and LCC sensitivity to reliability. 
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Figure 17. ETD3 ELR and LCC sensitivity to reliability. 


VIII. CONCLUSIONS AND RECOMMENDATIONS 


NASA has an increasing determination to once again be the forerunner in space endeav- 
ors. Permanent lunar bases and Space Station Freedom are just two concepts that NASA envi- 
sions for the next century. However, these concepts would require transportation systems 
responsible for cargo or logistics delivery, as well as transfer of personnel to and from these 
installations. The architecture studied in this report is one possible transportation system to 
satisfy these requirements. 

Most of the vehicles that comprise the architecture are proposed. The only existing 
vehicle with similar subsystems has not been operational long enough to provide reliability data 
that would support proposed vehicles. As shown in this study, a sensitivity analysis model is a 
valuable tool to evaluate the architecture. 

The purpose of this report was not to certify or annul any particular reliability estimate of 
the vehicles, but to reveal the sensitivity of the architecture to varied reliability allocations. 

It has been shown that effective launch rate is directly related to vehicle reliability. It has 
also been shown that both life cycle cost and failure cost are inversely related to reliability. Fur- 
thermore, there is evidence that vehicles in the architecture with complete autonomy, such as 
Titan IV, will more likely achieve the planned launch rate. Moreover, the ELR/reliability rela- 
tionships for ETD2 and ETD3 suggest that vehicles with low flight rates are scarcely affected by 
reliability. Low flight rates present fewer chances for failure. 

At this stage of NASA's analysis of this and other architectures, two considerations must 
be taken into account before making recommendations: 
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1. Funding is a limiting factor on how much the architectures may be modified. For 
example, to bring new elements on line earlier than scheduled may enhance the performance of 
the system; however, this action may also prove to be more costly than the present funding will 
allow. 


2. The plans and designs of the architecture at this point are flexible. A fixed program 
with fixed funding presents less of a challenge to design, develop, and implement. 

With this in mind, recommendations are somewhat constrained. However, facts revealed during 
the analysis of this architecture will be presented. 

It is intuitively obvious from the results of section VII that it would be to NASA's 
advantage to look closely at the human risk involved with a vehicle's failure to respond to 
emergencies or deliver logistics in a timely fashion. Even at an assumed reliability of 99 percent 
there will be failures. 

The cost activity generated by the model is inversely related to reliability. This suggests 
that in order to lower costs one must design a very reliable vehicle. Although it is impossible to 
design a vehicle to meet a specified reliability value exactly, a system may be designed for 
reliability. 

Blanchard and Fabrycky 3 explain that reliability is an inherent characteristic of design and 
must be an integral part of the overall systems engineering process. Reliability requirements are 
defined in conceptual design, reliability analyses and predictions are accomplished throughout 
preliminary and detail system/product design, reliability is considered in formal design reviews, 
and reliability testing is accomplished as part of system test and evaluation. Thus, reliability 
(along with other major design parameters) is considered throughout the system life cycle and is 
particularly relevant during the early phases of system design and development. 

Therefore, the only practicable recommendations regarding lowering costs are as outlined 
in the aforementioned book. The objective is to plan a program effort that will assure reliability 
involvement throughout all aspects of system design and development, production or 
construction, and system utilization. A reliability program plan is usually prepared at program 
inception and may be included as part of the system engineering management plan. 
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RELIABILITY ALLOCATION PROCEDURE 

This procedure was developed by Dr. James W. Steincamp, 
Chief, Operations Analysis Branch, Preliminary Design 
Office, NASA-Marshall Space Flight Center. 

The main propulsion system (mps) reliabilities varied 
among vehicles: 

STS (3 SSME'S) = 0.9936 (1:156) 

PLS/ETD1 (5 STME'S INCLUDING 2 SUSTAINERS ) = 0.9957 
( 1 : 233 ) 

ETD2 (17 STME'S) = 0.9492 (1:20) 

ETD3 ( 24 STME'S) = 0.9398 (1:17) 

The corresponding odds (chances of failure) were calculated 
as follows 

odds=l/( 1-R) 

The STS common elements and unique elements were assumed 


Common 


SRB's 

1:150 


R=0 . 9933 

avionics & 

other = 

1 : 400 

R=0 . 9975 

Unique 




avionics & 

other = 

1 : 500 

R = 0 . 9980 

operations 

s 

1 : 400 

R=0 • 9975 



With these values the corresponding reliabilities were 
calculated as follows 

R = 1-1 /odds 

The common/unique element subsystems have a serial 
relationship, the reliability calculation for STS common 
elements then is, 

STS Common Element Reliability=R(mps)*R(srb)*R(a&o) 

The elements also have a serial relationship and the system 
reliability is calculated as, 

STS System Reliability=R( Common )*R( Unique ) 

The elements are mutually exclusive and non-independent. 
Therefore, the probability of an element failure is 
calculated as in this example of the probability of a common 
failure . 

P( common) = 1 — R ( common element) 

1-R(common element )+( 1 -R( unique element)) 

The probability that a particular subsystem will fail is 
calculated as in this example of the probability of an SRB 
failure . 
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P(SRB) 


l-R(SRB) 


1 — R ( Common Element) 

The transition from 0.97 to 0.98 to 0.99 is simple. 

Since 0.97 is 1:33 and 0.98 is 1:50 and 0.99 is 1:100 (see 
above equation for calculating odds), the relationship 
between them is obvious. The relationship between 0.98 and 
0.97 is 33/50 and the relationship between 0.99 and 0.98 is 
100/50. Therefore to make the transition from 0.98 to 0.97 
you can simply multiply the 0.98 odds by 0.667 and likewise 
to make the transition from 0.98 to 0.99 you can multiply 
the 0.98 odds by 2. 
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